Optimal Convergence Rates for the Orthogonal Greedy Algorithm

We analyze the orthogonal greedy algorithm when applied to dictionaries $\mathbb {D}$ whose convex hull has small entropy. show that if metric entropy of decays at a rate notation="LaTeX">$O\left({n^{-\frac {1}{2}-\alpha }}\right)$ for notation="LaTeX">$\alpha > 0$ , then converges same on variation space . This improves upon well-known {1}{2}}}\right)$ convergence in many cases, most notably corresponding shallow neural networks. These results hold under no additional assumptions dictionary beyond decay its hull. In addition, they are robust noise target function and can be extended rates interpolation spaces norm. empirically predicted obtained networks with Heaviside activation two dimensions. Finally, we these improved sharp prove negative result showing iterates generated by cannot general bounded norm